Motor vehicle

ABSTRACT

A motor vehicle, having a plurality of image recording devices arranged across the entire vehicle for recording images of the vehicle environment, includes a control device for generating an image representation showing the 360° environment around the motor vehicle on the basis of the images. The motor vehicle further includes a display device for displaying the image representation. The control device is designed to generate an additional partial image representation showing the area below the motor vehicle on the basis of images recorded, and to integrate the partial image representation into the displayed image representation for outputting an overall image representation. At least one sensor device is provided which detects the area below the motor vehicle and communicates with the control device. The overall image representation can be modifiable at least in the area of the partial image representation on the basis of the sensor information.

TECHNICAL FIELD

The disclosure relates to a motor vehicle, having a plurality of imagerecording devices arranged across the entire vehicle for recordingimages of the vehicle environment, a control device for generating animage representation showing the 360° environment around the motorvehicle on the basis of the images, and a display device for displayingthe image representation, wherein the control device is designed togenerate an additional partial image representation showing the areabelow the motor vehicle on the basis of images recorded before theimages, on which the displayed image representation is based, and tointegrate the partial image representation into the displayed imagerepresentation for outputting an overall image representation.

BACKGROUND

Modern motor vehicles have the option, if desired, of generating animage representation showing the 360° environment around the motorvehicle and outputting it on a suitable monitor. A system that makesthis possible is frequently called “top view” or “surround view” or thelike. As a rule, it uses four cameras, namely one camera arranged on thefront of the vehicle and one on the rear of the vehicle, and one cameraeach arranged on the side, for example, on the side-view mirrors. Eachcamera provides separate images or image data, which are recorded andprocessed or computed jointly by a control device in order to generateand subsequently output a virtual top view, i.e., a two-dimensional topview of the motor vehicle and its environment, or a virtualthree-dimensional view with a three-dimensional vehicle model asself-representation and possibly a freely selectable perspective. Suchan image of the environment consequently enables the driver to recordthe entire vehicle environment to the extent that the individual camerasrecord toward the side.

The currently recorded images allow for a display of the current vehicleenvironment on the side of the vehicle, which can be viewed virtually“live.” However, this is not the case for the area below the vehiclebecause no camera is arranged there. Already known systems frequentlyoperate in such a way that the area below the vehicle is displayed as arectangle which is either black or shaded or also colored to match theenvironment. This area is also called “ground plane” and is sometimeslarger, e.g., in the case of cars, in the vehicle longitudinal directionthan the vertical projection of the vehicle, since the front and rearcameras only detect the ground in front of and behind the vehicle at acertain distance because the respective bumper limits the downward viewand creates a blind spot.

In addition to displaying the area as a colored or shaded rectangle, itis also known to fill this area, i.e., the “ground plane,” with sampledimage or video data, i.e., to generate and integrate into the imagerepresentation a partial image representation using images that wererecorded before the currently recorded images or before the images, onwhich the displayed image representation is based. This means that,depending on the direction of travel, image data are recorded in frontof or behind the vehicle and buffered, so that the current environmentbelow the vehicle can be effectively recalculated on the basis of thepreviously recorded image data and integrated as a partial imagerepresentation. For the driver, this results in an overall imagerepresentation from the effectively current image representation createdon the basis of the last recorded images, and the calculated, sampledpartial image representation computed on the basis of image datarecorded prior thereto. Filling the “ground plane” with such sampleddata is known, for example, from DE 10 2016 208 369 A1.

In this context, it is also known to switch, for example, thethree-dimensional vehicle model to (semi-)transparent, if necessary,thus providing a view under and behind the schematically displayedvehicle model. This is expedient when driving on difficult ground orwhen a charging plate for electric/hybrid vehicles is to be targetedunder the vehicle floor, or generally for improving the view in the 3Dscene, e.g., of objects lying behind the vehicle from the perspective ofthe virtual camera.

The problem with such a filling of the “ground plane” with bufferedimage data, i.e., with the display of such a partial imagerepresentation, and possibly also the transparent vehicle display, isthat the availability of the buffered data is low. When driving slowlyor at a standstill, they have to be deleted again after a short periodof time because it is not known whether, temporarily, there are objectsunder the vehicle that were not previously visible. For example, a ball,a cat, or a child may have found its way under a vehicle driving orstanding in a garage, which is possible due to the large temporal offsetbetween the images or image data used for creating the partial imagerepresentation and the current display time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a schematic diagram of a motor vehicle, according tosome embodiments of the present disclosure.

FIGS. 2 to 4 illustrate image representations below a motor vehicle,according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

This problem is addressed by the present disclosure, providing a motorvehicle improved in the respect.

In some embodiments, in a motor vehicle of the initially described type,at least one sensor device is provided which detects the area below themotor vehicle and communicates with the control device, wherein theoverall image representation consisting of image representation andpartial image representation is modifiable at least in the area of thepartial image representation on the basis of the sensor information.

According to some embodiments of the present disclosure, at least onesensor device is used to monitor the area below the motor vehicle todetermine whether the area is free from obstacles or objects, or whetheran object is located below the vehicle at the time of the monitoring.Ultimately, the use of the buffered image data and thus also thegenerating of the partial image representation or the output of theoverall image representation depends on the detection result. If noobject is detected under the vehicle, images or image data, which showthe area below the vehicle and were recorded significantly earlier, canalso be used to determine the partial image representation showing the“ground plane,” i.e., their usability was ultimately made possible orreleased via the sensor detection because it is ensured that there isultimately no change in the situation below the vehicle between the timethe “old” sampled image data were recorded and the current time. Thismeans that the availability and usability of older sampled image data issignificantly improved due to the sensor monitoring and recording of thespace below the vehicle.

If, on the other hand, the sensor device detects an object under thevehicle, for example, a ball, a toy, etc., the previously recorded imagerepresentations cannot be used or can only be used to a limited extentfor a “ground plane” display; as a result, the image representation isdisplayed in a modified manner at least in the area of the partial imagerepresentation when compared to the case without obstacle detection. Dueto this modified image representation, the driver recognizes that adisplayed partial image representation is either not reliable or thatthe area below the vehicle must in any case be checked. The driver thusreceives information about the obstacle or object situation below thevehicle via the modified overall image or partial image representation.

The sampling preferably runs permanently, so that there is always theoption of filling the “ground plane,” for example, in a top viewdisplay. The sampling mode is therefore always activated. If thesampling is not already running, the driver has the option, for example,of having a 360° representation of the environment displayed. If thedriver selects the representation option, the system can automaticallyswitch to the “ground plane” display mode, i.e., the partial imagerepresentation is automatically activated with the option of parallelsensor monitoring. Alternatively, the representation option for the“ground plane” can also be selected separately from the 360°representation of the environment.

A central element is the at least one sensor device for recording thearea below the vehicle. Such a sensor device can be installed as anadditional system component for the image representation systemaccording to some embodiments of the present disclosure. Alternatively,a sensor device that is already provided and assigned to anotherassistance system can also be used. For example, such underbody sensorsare already provided in vehicles designed for (partially) automateddriving to use them, for example, to carry out release checks forautomated starting or the like. Such a sensor or a plurality of suchsensors can now also be integrated into the 360° image representationsystem and consequently serve a further purpose, namely to decide on theusability of the older, buffered image data for determining the “groundplane” image representation. The basically usable sensor device can beany sensor device, for example, an ultrasound sensor, optical sensors orthe like, wherein this list is not exhaustive and limiting.

If an object is recorded below the vehicle, different options withregard to the use of this information are conceivable. According to afirst alternative of the disclosure, the control device can be designedto delete the images used for the partial image representation and todisplay the image representation without a partial image representationwhen an object is detected below the vehicle. In this alternative, thebuffered image data provided for the partial image representation aredeleted when an object is detected below the vehicle, i.e., no partialimage representation takes place because these image data do notcorrespond to or show the actual situation below the vehicle. The imagerepresentation is then displayed without a partial image representation.This way, the driver can already be informed about the fact of an objectbelow the vehicle; however, instead of the partial image representationin this area, a text or a warning symbol, i.e., a warning informationwithin the image area provided for the partial image representation, canpossibly also be displayed to indicate an obstacle, and/or an acousticwarning information can also be output on the basis of the sensorrecording. However, if no object is detected, the image representationis naturally output with the partial image representation created on thebasis of the sampled image data. Deleting does not have to go hand inhand immediately with the recording of the object. Instead, it is alsoconceivable that, after an object has been detected, it is first checkedwhether something has changed and what has changed spatially ortemporally under the vehicle in order to make it possible, e.g., torecognize that the vehicle is driving, e.g., over a charging plate thatis desirable to be seen in the image in order to maneuver accordingly.It is also conceivable to use the distance and the time covered tocompare whether an object determined by the underbody sensor systemalready existed before the vehicle drove over it and thus migrated intothe “ground plane” as a result of the driving. If such a situation isrecognized, the buffered images can still remain saved despite objectdetection, and can be deleted at a later time, e.g., after the vehiclehas been positioned and stopped.

According to a second alternative of the disclosure, it is conceivablethat the control device is designed to display the image representationwith a visually marked partial image representation when an object isdetected below the vehicle. In this case, despite the detection of anobject based on the “old” buffered image data, the partial imagerepresentation is generated and also integrated into the overall imagerepresentation, but it is marked visually, so that this visual markingsignals to the driver that this partial image representation does notcorrespond to the actual situation below the vehicle.

The visual marking can be done in different ways, for example, thepartial image representation can be highlighted in color. For example,it can be colored, especially in red as a signal color. It is alsoconceivable to display the marking in a flashing manner, for example, bydisplaying and hiding it, or by displaying it in a swelling andsubsiding manner (“glow” display). These various visual marking ordisplay options are also listed by way of example and not limitedthereto.

In some embodiments, the control device is designed to automaticallydetect the sensor information when the speed falls below a speedthreshold, particularly during a standstill. It is not necessary for thecontrol device to detect sensor information during normal travel becausea “ground plane” representation is not required during normal travel.Instead, the detection takes place primarily during slow travel orstandstill, so that the control device only requires correspondingsensor information in such events. According to some embodiments of thepresent disclosure, the control device automatically detects the sensorinformation when the speed falls below a corresponding speed threshold,for example, 10 km/h, but especially when the vehicle is at astandstill, since this provides a driving situation, in which image databuffering is expedient because a “ground plane” representation ispossibly desired at a later point in time. If the vehicle is parked, forexample, in the garage or in a parking lot, the sensor information canbe detected as soon as the ignition is switched on or the vehicle isstarted. In the driving situations described, it is also conceivablethat the control device also automatically switches on the sensor devicefor the detection, provided that it is currently inactive. The measuresdescribed above thus ensure that the sensor information is alwaysavailable when it might be needed.

In addition to the motor vehicle, the present disclosure further relatesto a method for operating a motor vehicle, comprising a plurality ofimage recording devices distributed around the vehicle for recordingimages of the vehicle environment, a control device for generating animage representation based on the images showing the 360° environmentaround the motor vehicle, and a display device for displaying the imagerepresentation, wherein the control device generates an additionalpartial image representation showing the area below the motor vehicle onthe basis of images which were recorded before the images, on which thedisplayed image representation is based, and inserts the partial imagerepresentation into the displayed image representation for outputting anoverall image representation. The method is characterized in that thearea below the motor vehicle is detected by at least one sensor devicecommunicating with the control device, wherein the control devicemodifies the overall image representation of image representation andpartial image representation at least in the area of the partial imagerepresentation on the basis of the sensor signals.

As already initially described, there are various options for thismodification. When an object is detected below the vehicle, the imagesused for the partial image representation can be deleted by the controldevice and the image representation can be output without the partialimage representation. Alternatively, it is also conceivable for thecontrol device to output the image representation with a visually markedpartial image representation or with warning information within theimage area provided for the partial image representation when an objectis detected below the vehicle. In such case, the partial imagerepresentation can be highlighted in color or displayed in a flashingmanner.

In some embodiments, the control device automatically detects the sensorinformation when the speed falls below a speed threshold, particularlyduring standstill, or when the vehicle is started, wherein the sensordevice for the detection is possibly also switched on automatically forthe detection, provided that it is not activated.

Further advantages and details of the present disclosure shall becomeapparent from the embodiments described below and by means of thedrawings.

FIG. 1 shows a motor vehicle 1 according to some embodiments of thepresent disclosure, comprising four image recording devices 2, onearranged on the front and one on the rear of the vehicle, and one imagerecording device each on the side of the vehicle, for example, on theside-view mirrors, wherein only three image recording devices are shownin FIG. 1. They record environmental image data from the environment ofthe vehicle and transmit them to a control device 3 which determines animage representation of the 360° environment around the motor vehicleand outputs it to a display device 4 when the driver, for example, hasselected a corresponding 360° environment image mode on the outputdevice 4, for example, a touchscreen monitor.

In order to also be able to display the area below the motor vehicle inthe image representation, the control device 3 is able to calculate thisarea within the framework of a determined partial image representationusing images or image data recorded earlier and to display it in theimage representation, resulting ultimately in one overall imagerepresentation which comprises the 360° environment image representationoutside the vehicle and the partial image representation integratedtherein relating to the area below the vehicle.

In order to ensure the availability of the older image data for as longas possible, at least one sensor device 5 is provided which communicateswith the control device 3 and monitors the area below the motor vehicle1 for any obstacles, for example, a ball which rolled under it, a toy orthe like. These objects were not under the vehicle at the time when theearlier images, on which the partial image representation was based,were recorded and can be problematic for the drive if the partial imagerepresentation is determined on the basis of the older images, which donot show the actual situation, and which is thus incorrect.

If the control device 3 is in the corresponding mode in order togenerate the image representation and also to generate the partial imagerepresentation, the area below the vehicle is monitored via the sensordevice 5 and, depending on whether an object is determined or not, thecorresponding generating and output mode takes shape.

FIG. 2 shows a schematic diagram of an overall image representation 6,comprising the image representation 7, which shows the area outside themotor vehicle 1, which is only shown in dotted lines, i.e., effectivelypartially transparent, and which is detected on the basis of the imagescurrently recorded.

The partial image representation 8 is displayed effectively within thedotted area showing the motor vehicle silhouette, wherein in this case,the partial image representation is determined on the basis ofpreviously recorded images which were recorded at the time the vehicledrove over the area, which is now located below the motor vehicle 1,because the sensor device has not detected an object under the motorvehicle. These image data are verified on the basis of the sensorinformation insofar as they still show the actual situation below themotor vehicle, even though they are older, i.e., they can be used, eventhough they are buffered.

The situation is different in the representation according to FIG. 3. Italso shows an overall image representation 6 in the form of a schematicdiagram, with the image representation 7 showing the 360° environmentand a displayed partial image representation 8. In this case, it isassumed that the sensor device 5 has detected an object below thevehicle. A partial image representation 8 is determined on the basis ofimage data recorded earlier. However, they no longer correspond exactlyto the current situation and are therefore shown visually marked. In theexample, this is indicated by the significantly larger line thickness.For example, the partial image representation is output in red but canalternatively also be highlighted in red, or flash, etc. Any visualmarking is conceivable as long as it can be immediately detected by thedriver. The visual highlighting signals to the driver that the partialimage representation does not show the current status below the vehicleand that an object has now been detected below the vehicle, so that thedriver must under no circumstances rely on the partial imagerepresentation and instead check the area below the vehicle and removethe object. An acoustic warning can also be output.

FIG. 4 shows a third variation of an overall image representation 6. Italso shows the image representation 7 relating to the 360° environment,which was determined on the basis of the current images. However, thearea within the silhouette of the motor vehicle 1 is not filled, i.e.,no partial image representation was determined because an object wasdetected below the vehicle by the sensor device 5. Instead, a warningsymbol 9 is shown in this area, which informs the driver of thepotentially dangerous situation.

The sensor device 5 can be a separate sensor device that is assignedexclusively to this assistance system. Alternatively, it can also bepart of an underbody sensor system, which is used, for example, insemi-autonomously and fully autonomously driving motor vehicles, formonitoring the underbody space and for releasing an automatic startingor the like.

Finally, the control device 3 can be designed for automatic detection ofthe sensor information when the speed falls below a speed threshold, forexample, 10 km/h or 5 km/h or at a standstill. It is thus ensured that,whenever there is the subsequent possibility of displaying a partialimage at a later point in time, corresponding sensor information is alsorecorded. In addition, the control device 3 can naturally also bedesigned to automatically record the sensor information as soon as thevehicle is about to be restarted, i.e., particularly in such case, thesensor information is also recorded immediately, and it can be ensuredthat the validity of the older image data is checked for the partialimage representation. If necessary, the detection mode of the sensordevice can basically also be switched on via the control device,provided that it is initially inactive.

1.-10. (canceled)
 11. A motor vehicle, comprising: a plurality of imagerecording devices arranged across the motor vehicle and configured torecord images of a vehicle environment; a display device configured todisplay an image representation showing the vehicle environment; asensor device configured to detect an area below the motor vehicle; anda control device configured to: generate the image representation usingthe images of the vehicle environment; generate an additional partialimage representation showing the area below the motor vehicle on thebasis of images recorded before the images used to generate the imagerepresentation; integrate the additional partial image representationinto the image representation to output an overall image representation;communicate with the sensor device; and modify the overall imagerepresentation, in at least a portion of the additional partial imagerepresentation, on the basis of a sensor information from the sensordevice.
 12. The motor vehicle according to claim 11, wherein the controldevice is further configured to: delete the images used for theadditional partial image representation when an object is detected belowthe vehicle; and output the overall image representation without theadditional partial image representation.
 13. The motor vehicle accordingto claim 11, wherein the control device is further configured to: outputthe overall image representation with the additional partial imagerepresentation visually marked, or replaced by a warning information,when an object is detected below the vehicle.
 14. The motor vehicleaccording to claim 13, wherein the additional partial imagerepresentation is highlighted in color or in a flashing manner.
 15. Themotor vehicle according to claim 11, wherein the control device isfurther configured to automatically detect the sensor information when aspeed of the motor vehicle falls below a speed threshold, when the motorvehicle is at a standstill, or when the motor vehicle is started.
 16. Amethod for operating a motor vehicle, comprising: recording images of avehicle environment through a plurality of image recording devicesarranged across the motor vehicle; generating, by a control device, animage representation showing the vehicle environment on the basis of theimages; displaying, by a display device, the image representation;generating, by the control device, an additional partial imagerepresentation showing an area below the motor vehicle on the basis ofimages recorded before the images used to generate the imagerepresentation; integrating the additional partial image representationinto the image representation to output an overall image representation;detecting the area below the motor vehicle by a sensor devicecommunicating with the control device; and modifying, by the controldevice, the overall image representation, in at least a portion of theadditional partial image representation, on the basis of a sensorinformation from the sensor device.
 17. The method according to claim16, further comprising: deleting, by the control device, the images usedfor the additional partial image representation when an object isdetected below the vehicle; and outputting the overall imagerepresentation without the partial image representation.
 18. The methodaccording to claim 16, further comprising: outputting, by the controldevice, the overall image representation with the additional partialimage representation visually marked or replaced by a warninginformation when an object is detected below the vehicle.
 19. The methodaccording to claim 18, further comprising: highlighting the additionalpartial image representation in color.
 20. The method according to claim18, further comprising: displaying the additional partial imagerepresentation in a flashing manner.
 21. The method according to claim16, further comprising: detecting automatically, by the control device,the sensor information when a speed of the motor vehicle falls below aspeed threshold, when the motor vehicle is at a standstill, or when themotor vehicle is started.